There is an increase in demand for Dynamic Random Access Memory (DRAM) or main memory in environments such as data centers for application performance. DRAM can be located directly in a server or shared in networked instances of in-memory database applications such as, for example, Memcached, Redis, and Mcperf.
A recently developed practice in the storage industry is to use a low latency Solid-State Drive (SSD), such as a Non-Volatile Memory express (NVMe) SSD, to extend and/or replace a portion of the main memory or DRAM of a host. For example, an entire storage capacity of an SSD (e.g., 750 GB) can be made visible as virtual memory to a host's Operating System (OS) and at least one application executed by the host (e.g., Memcached, Redis, Apache Spark). In some cases, an NVMe SSD can replace 80% of the DRAM that would otherwise be needed. This can provide a lower overall cost without a significant loss in performance for DRAM intensive operations. The host may then include a smaller DRAM.
A memory pool can include a DRAM with a capacity of 10% to 20% of the total capacity of the NVMe SSD for caching frequently accessed data (i.e., “hot data”). The DRAM and the memory of the NVMe SSD can be shared among multiple servers or hosts. Swapping software such as Intel-Memory Drive Technology (IMDT) can execute at the host, and can work with a Memory Management Unit (MMU) of the CPU to present the entire capacity of the NVMe SSD as virtual memory. The NVMe SSD then appears as a virtual memory block device to a host OS and to one or more applications executing at a host.
As an application accesses the virtual memory, some of the accesses will result in a DRAM hit, meaning that the data for the access has already been cached in the DRAM. Other accesses will result in a DRAM miss, meaning that the data has not already been cached in the DRAM. In the case of a DRAM miss, space is created in the DRAM by evicting or flushing a 4K page to the NVMe SSD and reading the requested data from the NVMe SSD and loading it into the DRAM of the memory pool. The virtual memory mapping is then changed by the swapping software. This process can be referred to as a virtual memory exception or page fault. When a page fault occurs, the application thread that encountered the page fault is suspended while another application thread continues processing. After the page fault has been handled through the loading of the new 4K page into the memory pool's DRAM, the suspended thread is resumed.
Although the latencies associated with such page faults are tolerated by multi-threaded applications, a symmetric or similar read and write performance in terms of read Input/Output Operations Per Second (IOPS) and write IOPS is needed in writing the evicted page to the NVMe SSD and reading the new page into the DRAM of the memory pool. Such performance can be obtained with relatively expensive Storage Class Memories (SCMs), as in the example of Intel's Optane storage device. However, flash memories, such as NAND memories, generally have much lower write performance (e.g., write IOPS) as compared to read performance (e.g., read IOPS), such as a three times lower performance (e.g., IOPS) for random writes than for random reads. The time to perform the page eviction or flush to the NVMe SSD discussed above becomes a limiting factor for handling page faults with such NAND memories. In addition, evicting 4K pages to a NAND memory in an SSD can create numerous write operations that deteriorate the endurance or usable life of the NAND memory.